Imagers typically consist of an array of thousands or even millions of pixel cells organized into rows and columns. Each pixel cell contains a photosensor (e.g., a photodiode) that is configured to produce an electrical signal corresponding to the intensity of light impinging on it when an image of a scene is focused on the array by one or more lenses. Each signal can be converted to a digital pixel value (i.e., a number associated with a pixel cell), usually with small numbers representing lower light intensity (i.e., darker areas of the scene) and larger numbers representing higher light intensity (i.e., brighter areas of the scene). A digital image can be formed by organizing the pixel values into rows and columns corresponding to the arrangement of their respective pixel cells in the pixel array. The digital image can be stored in a memory, displayed on a monitor, manipulated by software, printed, or otherwise used to provide information about the scene.
Several kinds of imagers are generally known. Complementary metal-oxide-semiconductor (“CMOS”) imagers and charge coupled device (“CCD”) imagers are among the most common. CMOS imagers are discussed, for example, in U.S. Pat. No. 6,140,630, U.S. Pat. No. 6,376,868, U.S. Pat. No. 6,310,366, U.S. Pat. No. 6,326,652, U.S. Pat. No. 6,204,524, and U.S. Pat. No. 6,333,205, all assigned to Micron Technology, Inc.
For an imager to capture a color image, its pixel cells must be able to separately detect values of colored light, for example, red (R) light, green (G) light, and blue (B) light. A color filter array is typically placed in front of the array of pixel cells so each pixel cell measures only light of the color of its respective filter. The most common type of color filter array, often referred to as a “Bayer color filter array,” is described in U.S. Pat. No. 3,971,065. As shown in FIG. 2A, Bayer color filter arrays consist of alternating red (R) and green (G) filters in a first row and alternating green (G) and blue (B) filters in a second row. This pattern is repeated throughout the array. Thus, in an imager employing a Bayer color filter array, one-half of the pixel cells are sensitive to green light, one-quarter are sensitive to red light, and the remaining one-quarter are sensitive to blue light.
To provide a color image, however, red, green, and blue values are required for every pixel. Thus, the two “missing” color values at each pixel location must be estimated, usually by reference to other nearby pixel values. For example, the pixel at row r+1 and column c+1 in FIG. 2A is sensitive to red light, as indicated by the “R” within the pixel cell. Blue and green color values for this pixel can be estimated based on color values of nearby pixels. This estimation process is often referred to as “demosaicing.”
Several demosaicing methods are known in the art. The simplest methods consider each color separately. For example, missing green pixel values are estimated based on nearby green pixel values. Bilinear interpolation is one such method and provides good results when applied to grayscale images. However, bilinear interpolation in color imaging often results in many visible artifacts (e.g., zipper effect or artificially colored or blurred edges).
Bilinear interpolation and other standard interpolation methods treat each color independently. However, color images often exhibit a high degree of correlation among color channels. Several methods have been proposed to use this correlation to more accurately estimate missing color pixel values. One such method involves analyzing pixels around each missing color pixel value to determine whether the region exhibits a preferred interpolation direction. For example, in one implementation, pixel values from to the left of a missing color pixel value are subtracted from pixel values to the right of the missing color pixel value to determine a horizontal gradient score. Similarly, a difference between pixel values above and below the mixing color pixel value is computed to determine a vertical gradient score. By comparing the scores, these “edge-directed” methods attempt to determine whether the missing color pixel value should be interpolated horizontally, vertically, or both. In general, the interpolation direction is chosen to interpolate along edges rather than across edges.
Wrongly selecting the interpolation direction is a major source of error in edge-directed demosaicing methods. For example, if interpolation direction changes abruptly along an edge, a zipper effect occurs. Similarly, if an interpolation direction in a region of an image is random or difficult to determine, color aliasing occurs. Thus, there is a need in the art for an improved demosaicing method that provides, among other things, more accurate interpolation direction selection. U.S. application Ser. No. 11/873,123 discloses systems and methods for demosaicing image data with more accurate interpolation direction selection.
After demosaicing, three or four color values are typically associated with each pixel, depending on the color space used by the imager or processing software. RGB and YCbCr are two common color spaces. In the RGB color space, also referred to as the “Bayer domain,” a red value (R), a blue value (B), and a green value (G) are associated with each pixel. In the YCbCr color space, a luminance (i.e., brightness) value (Y) and two chrominance (i.e., color) values (Cb and Cr) are associated with each pixel.
Sometimes demosaiced color pixel values do not accurately reflect the true color in a corresponding part of a scene. For example, a pixel corresponding to the edge of a white region in a scene might appear colored rather than white. Digital image processing can be used to correct such errors, which are also referred to as “color artifacts.” Aliasing artifacts are one common kind of color artifact. Aliasing occurs when detail in a scene exceeds the sampling frequency of the imager, for example, when the lines of detail in an image exceed the number of rows of pixels in the pixel array of an imager. Aliasing can result in color artifacts along edges of details in an image and especially, though not exclusively, along edges involving an abrupt black-to-white transition. The tendency of lenses to refract different wavelengths of light differently is another common cause of color artifacts. Color artifacts may also appear as isolated colored “dots,” sometimes comprising only a single wrongly-colored pixel, in an image. These isolated colored dots can be caused, for example, by a defect in the imager or an error during demosaicing.
Correcting the root causes of color artifacts requires additional hardware components or substitution of higher-quality hardware components, such as, for example, a pixel array with more pixels or lenses that refract varying wavelengths of light more evenly. These solutions are often impractical, particularly in low-cost imagers. Therefore, a less expensive method for reducing color artifacts in digital images, particularly one which does not require additional or higher-quality hardware, is desirable. U.S. application Ser. No. 12/025,926 discloses such systems and methods for reducing color artifacts in digital images.
While the '123 and '926 applications disclose systems and methods for improved demosaicing and color artifact reduction, there remains a perpetual demand for even more hardware-efficient solutions.